The present invention deals with named entity recognition. More specifically, the present invention deals with modeling user patterns to enhance the recognition of named entities.
A named entity (NE) is a specific linguistic item, such as a proper name, the name of a company, an email address, a location, etc., which is treated as one unit by an application. Named entity recognizers are known, and named entity processing is known to be an important stage of linguistic analysis.
NE recognition is currently done in a number of ways. Some approaches for NE recognition use list look up when NEs are fixed (or static) such as city names, country names, first names, company names, fixed terms like product names, etc. Other approaches use regular expressions and grammar rules that can combine syntactic information with a lexicon or list look up in order to recognize NEs. Most common approaches build finite-state recognizers directly from training data.
Personal information management (PIM) continues to be a major application area of automatic speech recognition. In PIM systems, NE recognition is an important task. For example, speech enabled electronic mail programs rely on NE recognition. Processing in these applications requires the recognition of many NEs, such as email recipients. Similarly, some PIMs involve voice dialing of telephones, scheduling, etc. which also require NE recognition.
NE recognition, in general and in the domain of speech recognition engines, poses a number of problems. First, NE users may have access to a very large number of names or other terms that constitute NEs. For example, there are many distinct proper names in any given corporation or community. This leads to a very large perplexity, which in turn leads to a large error rate in the NE recognition task.
Another difficulty involves names not in the standard dictionary. It is extremely difficult to create a dictionary that contains every name in the world, or even all English sounding names. Even if such a dictionary exists, because of the vast number of possible pronunciations it would contain, using such a dictionary would significantly lower recognition accuracy.
Another difficulty presented in NE recognition is that there can be many different ways to refer to a single person. For example, different people refer to a single person in different ways. Some may call a person by his or her first name, while others may refer to the person with both the first and second names and still others may refer to the same person in different ways, such as using the first name and last initial or such as by using nicknames, etc. Moreover, a single person may refer to other people in multiple different ways. For example, a person may refer to one of his or her neighbors or coworkers using only the first name, while referring to another neighbor or coworker using only the last name.
Still another difficulty presented by NE recognition is ambiguity when only a first name is used. There are many common first names, and even if an NE recognizer (such as an automatic speech recognition system) correctly recognizes the word which forms the first name, the user may still be frustrated. For example, if the user speaks the word “David”, the automatic speech recognition system may accurately recognize the word “David” as a NE. However, in a large company there may be many people with the first name “David” and the user would then be required to hunt through a list of “David's” presented by the recognizer (and typically alphabetized by last name) to find the proper one.